As datasets grow larger and models more complex, machine learning increasingly requires distributing the optimization of model parameters over multiple machines. Existing machine learning algorithms are typically only applicable to controlled environments (such as data centers) where the data is properly distributed among machines, and high-throughput fiber networks are available.